[Storage] Add sdk/storage/internal shared module - #27437

sdk/storage/internal/structuredmsg/structured_message.go:461

• Replace the corrupted replacement character in this comment.

		// All segments done - emit trailer

sdk/storage/internal/structuredmsg/structured_message_test.go:1324

• The segment header is 10 bytes (SMSegmentHeaderSize), so this byte-layout comment has the wrong size.

	truncated := encoded[:truncateAt]

sdk/storage/internal/structuredmsg/structured_message.go:54

• numSegments can exceed the uint16 wire field (for example, 70,000 bytes with segmentSize == 1). The casts in the header and segment numbers then wrap, so this function returns a malformed payload that cannot round-trip. The azfile and azdatalake copies already increase segmentSize in this case; retain that protection in the consolidated implementation.

	numSegments := totalDataLen / segmentSize
	if totalDataLen%segmentSize != 0 {
		numSegments++

sdk/storage/internal/structuredmsg/structured_message.go:312

• Converting contentLen to int before division corrupts lengths above MaxInt on 32-bit builds. For example, a 3 GiB stream becomes a negative segment count, causing the encoder to emit only framing with a wrapped count instead of the content. Keep length and segment-count arithmetic in int64, converting only the final bounded segment count after enforcing the uint16 limit.

// NewSMEncoder creates a streaming encoder that wraps the given content source.
// contentLen is the total size of the content (must be known upfront for the SM header).
// segmentSize specifies the max segment size; use 0 for the default (4MB).

sdk/storage/internal/structuredmsg/structured_message.go:806

• A zero message length bypasses validation entirely, so changing an otherwise valid response header's length field to zero is accepted by the streaming decoder. The one-shot decoder correctly rejects the same payload, and zero cannot be a valid encoded length because framing alone is nonempty. Compare unconditionally using unsigned values to preserve the full wire range.

	}
}

sdk/storage/internal/structuredmsg/structured_message.go:820

• When the source returns the last framing bytes together with a non-EOF error, this branch discards that error and continues decoding. Segment-data reads explicitly propagate this case, but a transport error at a header/footer/trailer boundary can currently be reported as success. Preserve non-EOF errors even when the frame became full.

	d.state = decStateDone
	return nil

sdk/storage/internal/structuredmsg/structured_message_test.go:1198

• The segment header is 10 bytes (SMSegmentHeaderSize), not 6 bytes. Update the boundary description so it matches the calculation below.

	// Header (13 bytes) + segment header (10 bytes) + segment data (segSize bytes).

sdk/storage/internal/structuredmsg/structured_message_test.go:1301

• The segment header is 10 bytes, not 6 bytes; the comment currently contradicts SMSegmentHeaderSize used by the test.

	// Truncate after segment data, partway through the segment footer.

sdk/storage/internal/structuredmsg/structured_message_test.go:1321

• The segment header is 10 bytes, not 6 bytes; update this size breakdown to match the actual XSM framing.

	// Truncate after segment footer, partway through the message trailer.

sdk/storage/internal/structuredmsg/structured_message.go:206

• XSM/1.0 requires at least one segment, but the one-shot decoder accepts a crafted 21-byte message with numSegments == 0 and an empty CRC64 trailer. Reject zero here so malformed messages cannot bypass all segment validation.

	numSegments := binary.LittleEndian.Uint16(smData[11:13])

sdk/storage/internal/structuredmsg/structured_message.go:816

• This length check only compares bytes consumed through the trailer with the embedded length. If a valid encoded message is followed by extra bytes while retaining its original msgLen, the decoder reaches decStateDone without reading those bytes and accepts the body, even though XSM requires message length to match the complete HTTP body. Validate the underlying body's end (or pass its known content length into the decoder) before reporting success.

	// The consumed byte count must match the declared message length, so a stream that declares
	// fewer segments (leaving trailing bytes unvalidated) is rejected rather than silently accepted.
	if d.bytesRead != int64(d.msgLen) {

sdk/storage/internal/structuredmsg/structured_message.go:838

• The full-frame check runs before handling err, so a source returning the final framing bytes together with a non-EOF error (for example, a transient net.Error) has that error silently discarded. This defeats the retry behavior preserved in the segment-data path. Only suppress EOF when it accompanies a complete frame; propagate other errors first.

		if d.frameHave >= d.frameNeed {
			return true, nil
		}
		if err != nil {

sdk/storage/azblob/blob/client.go:518

• This newly added parallel path also leaves the retry reader and HTTP response body open when io.Copy fails. Close it before returning so failed range downloads do not leak connections.

			if _, err = io.Copy(shared.NewSectionWriter(writer, chunkStart+writerOffset, count), body); err != nil {
				return err
			}

sdk/storage/azblob/blob/client.go:393

• This writes the response ETag into the caller-owned AccessConditions. Reusing the same options after the blob changes then sends the stale injected If-Match and can fail unexpectedly; concurrent reuse can also race. Clone the nested condition structs before adding the per-download ETag.

		} else if o.AccessConditions.ModifiedAccessConditions == nil {
			o.AccessConditions.ModifiedAccessConditions = &ModifiedAccessConditions{IfMatch: dr.ETag}
		} else if o.AccessConditions.ModifiedAccessConditions.IfMatch == nil {
			o.AccessConditions.ModifiedAccessConditions.IfMatch = dr.ETag

sdk/storage/azblob/blob/client.go:592

• Skipping this block when downloaded == 0 leaves existing destination contents intact for an empty blob (or an empty range), despite DownloadFile promising to truncate the file to the downloaded size. Always compare and truncate, including to zero.

	if downloaded > 0 {

sdk/storage/azblob/blob/client.go:474

• When copying a block fails, this returns without closing the retry reader and its active HTTP response body. With concurrent downloads, repeated read/write failures can leak connections and exhaust transport resources; close the body on this error path.

This issue also appears on line 516 of the same file.

			if _, err = io.Copy(shared.NewSectionWriter(writer, chunkStart, count), body); err != nil {
				return err
			}

sdk/storage/internal/structuredmsg/structured_message.go:745

• The streaming decoder does not reject a zero segment count, unlike SMDecode. If an otherwise valid one-segment message has bytes 11–12 cleared, this decoder still parses segment 1, treats it as the last segment (1 >= 0), validates the CRCs and length, and accepts malformed metadata. Reject zero here and add the streaming equivalent of the one-shot validation test.

	d.numSegments = binary.LittleEndian.Uint16(buf[11:13])

sdk/storage/internal/structuredmsg/structured_message.go:875

• After a truncated or otherwise invalid header, Read moves the decoder to decStateError, so this condition no longer holds and DecodeResult returns a non-nil, partially initialized result even though the documented prerequisite was never met. Track successful header parsing explicitly (for example, with a boolean set only after parseHeader validates the header) and return nil until then; add a case using the existing truncated-body test.

	if d.state == decStateHeader {
		return nil

Previously missed (1) — in code that hasn't changed since the last review.

sdk/storage/internal/structuredmsg/structured_message.go:745

• Unlike SMDecode, the streaming decoder does not reject a zero segment count. If an otherwise valid one-segment message has bytes 11–12 cleared, this decoder still reads segment 1, treats 1 >= 0 as completion, validates both CRCs, and accepts the malformed message with NumSegments == 0. Reject zero here before entering the segment state.

	d.numSegments = binary.LittleEndian.Uint16(buf[11:13])